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ABSTRACT 


Indian army has always been subject to military attacks from neighbouring 
countries. Despite many surveillance devices and border security forces, the 
enemy finds a way to infiltrate deep into our borders. This is mainly because 
even now the surveillance in India is largely human-assisted. Therefore this 
automated surveillance can authenticate the authorized persons and alert 
everyone when an enemy intrusion is detected. In this, we proposed an in 
automated surveillance system that tackles the predicament of recognition of 
faces subject to different real-time scenarios. This model incorporates a 
camera that captures the input image, an algorithm to detect a face from the and 
input image, recognize the face using a convolution neural network along with 
transfer learning method, and verifies the detected person. The authorized 
person's name and details are stored in CSV format and then into the database. 
In case of any unauthorized person's face is detected the image of the intruder 
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along with time is stored in the database and warning signal is also given to 


alert the surrounding members in case of intrusion detection. 
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I. INTRODUCTION 

Indian Border has been facing many attacks from the foes, 
which cause lot of financial as well as mental & physical 
crisis. In the recent years, also numerous attacks have been 
took place in military field. So despite many surveillance 
devices and border security forces, the foes find a way to 
infiltrate deep into our borders. This attacks are mainly due 
to the surveillance in India is human-assisted. With 
consideration of such critical conditions we developed an 
automated surveillance system for military purposes. This 
automated surveillance system is implemented using 
Convolution Neural Network along with transfer learning 
.We can use this kind of technologies in the border area to 
keep tracking if any enemy intrusion enters the border. 
Thus, military attacks can be avoided. This model 
incorporates a camera that captures the input image, an 
algorithm to detect a face from the input image, recognize 
the face using a convolution neural network, and verifies the 
detected person. The face detection in this system is done 
through the Haar cascade classifier. Haar cascades make use 
of the image subtraction morphological process to detect the 
face. In this, the cascades of different images of the same 
person are taken and recorded in the database. The recorded 
images are using to train the CNN model. Here we have 
employed our CNN through transfer learning. The transfer 
learning models consist of several pre-trained models. One 
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such pre-trained model of VGG 16 is employed here. The 
model learns the facial features of the person images stored 
in the database. This trained and learned model is used in 
the surveillance system. Any other person’s face other than 
the faces in the database are detected and an intrusion alert 
signal is given. In case of authorized person’s face is detected 
the name and details are stored in CSV format and then 
stored into the database. In case of any unauthorized 
person’s face is detected the image of the intruder along with 
time is stored in the database. A warning signal is also given 
to alert the surrounding members in case of intrusion 
detection. 


II. RELATED WORK 

The concept of Face Recognition using Deep learning and 
edge computing technologies, which are used for efficient 
processing of huge amount of data with distinct accuracy 
was proposed by Muhammad Zeeshan, SaadHarous, 
Razilqbal [1]. They proposed an algorithm for face detection 
and recognition based on convolution neural networks 
(CNN), which outperform the traditional techniques. The 
concept of Face Recognition Based on Convolutional Neural 
Network was proposed by YakupDemir, OzalYildirim [2]. 
This method is implemented using modified Convolutional 
Neural Network (CNN) architecture by adding two 
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normalization operations to two of the layers. The 
normalization operation which is batch normalization 
provided accelerating the network. The concept of Face 
Detection and Tracking using OpenCV proposed by by 
KrutiGoyal, KartikeyAgarwal, Rishi Kumar [3].In this method 
they have developed a application for tracking and detecting 
faces in videos and in cameras which can be used for 
multipurpose activities. The concept of Eigen-faces for 
recognition was proposed by Mathew Turk and Alex Pent 
land [4]. In this method they developed a near-real time 
computer system that can locate and track a subject’s head, 
and then recognize the person by comparing characteristics 
of the face to those of known individuals. The concept of Face 
recognition using Eigen-face sand artificial neural networks 
was proposed by MayankAgarwal, Nikunj Jain, Mr. Manish 
Kumar and HimanshuAgrawal [5]. This paper presents a 


Il. METHODOLOGY 


methodology for face recognition based on information 
theory approach of coding and decoding the face image. 


The work in this paper has two methods.1)Face recognition 
using encoding technique 2)Face recognition using CNN and 
transfer learning method. Face recognition without using 
CNN is done by using encoding method. Here we will find all 
of the faces in the given image and label them and then we 
will compare the input image with the encoded file. The face 
distance is calculated to find the image that matches with the 
input image. The best match is found using the image with 
the smallest face distance. The label of the original image is 
displayed to detect the face of the given input image. Face 
recognition using CNN and transfer learning method is done 
by using the concept of transfer learning and using the pre- 
trained weights of VGG16 architecture, we can save both our 
time and resources. 


Face recognition is the problem of identifying or verifying faces in a photograph. 


3.1. Face recognition using encoding technique: 


Face recognition is often described as a process that first involves four steps; they are: face detection, face alignment, feature 


extraction, and finally face recognition. 
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Figure 1: Face recognition processing flow 


The description of each block is explained below: 


1. Face Detection. Locate one or more faces in the image and mark with a bounding box. 
2. Face Alignment. Normalize the face to be consistent with the database, such as geometry and photometrics. 
3. Feature Extraction. Extract features from the face using Haar feature extraction method, that can be used for the 


recognition task. 


4. Face Recognition. Perform matching of the face against one or more known faces in a prepared database using encoding 


method. 


3.2. Face recognition using CNN: 


Convolutional Neural Networks expect and preserve the spatial relationship between pixels by learning internal feature 
representations using small squares of input data. Feature are learned and used across the whole image, allowing for the 
objects in the images to be shifted or translated in the scene and still detectable by the network. VGG 16: 


VGG 16 was proposed by Karen Simonyan and Andrew Zisserman of the Visual Geometry Group Lab of Oxford University in 
2014 in the paper “Very deep convolutional networks for large-scale image recognition”. This model achieves 92.7% top-5 test 
accuracy on ImageNet dataset which contains 14 million images belonging to 1000 classes. The ImageNet dataset contains 
images of fixed size of 224*224 and have RGB channels. So, we have a tensor of (224, 224, 3) as our input. This model process 
the input image and outputs the a vector of 1000 values. 
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Figure 2: Architecture of VGG-16 
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Figure 3: Block diagram of face recognition using CNN & transfer learning method 


The description of each block is explained below: 


1. 


O22: 


3.3. 


3.4. 


3.5. 


3.6. 


Capturing using camera: Camera installed in army camp is used to capture the face of the persons who are present in the 
camp. 

Image processing: Facial detection and recognition algorithm is applied on the captured video frames. The detected face is 
cropped and stored for processing. The module recognizes the images of the face which have been registered manually 
with their names in the database. We use Open CV for all the image processing and acquisition operations. 
Convolutional layers: Convolutional layers are comprised of filters and feature maps. 


Filters: The filters are the “neurons” of the layer. The have input weights and output a value. The input size is a fixed 
square called a patch or a receptive field. 


Feature maps: The feature map is the output of one filter applied to the previous layer. A given filter is drawn across the 
entire previous layer, moved one pixel at a time. Each position results in an activation of the neuron and the output is 
collected in the feature map. 


Padding: The distance that filter is moved across the the input from the previous layer each activation is referred to as 
the stride. If the size of the previous layer is not cleanly divisible by the size of the filters receptive field and the size of 
the stride then it is possible for the receptive field to attempt to read off the edge of the input feature map. 


Relu: ReLU stands for rectified linear activation unit and is considered one of the few milestones in the deep learning 
revolution. It is simple yet really better than its predecessor activation functions such as sigmoid or tanh. 


Pooling layer: The Pooling layer is responsible for reducing the spatial size of the Convolved Feature. This is to decrease 
the computational power required to process the data through dimensionality reduction. Furthermore, it is useful for 
extracting dominant features which are rotational and positional invariant, thus maintaining the process of effectively 
training of the model. 


Fully connected layers: Fully connected layers are the normal flat feed-forward neural network layer. These layers may 
have a non-linear activation function or a softmax activation in order to output probabilities of class predictions. 


Database: The database stored the time and details of the detected person. We have implemented using MySQL database. It 
is an open source relational database. The prediction from the CNN is in the form of CSV which is the converted into a table 
format of rows and columns. If there is an unauthorized person the persons face is cropped and stored with time and a 
warning signal is given. 
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IV. EXPERIMENTAL RESULTS 

The final result of face recognition is shown in figure 4,5,6,7,8,9. figure 4 shows the keys and the values of the encoded image, 
figure 5 shows the accuracy plot we got accuracy percentage of 98.76, figure 6 shows the loss plot we gota loss percentage of 
9.79,figure 7 and 8 represents the face recognition of person, that is If the person is authorized then it will display the face of an 
authorized person under the green colour rectangle, else it will display unknown person in red rectangle, figure 9 shows how 
details are stored in the database. 


4.1. Face recognition using encoding technique: 

Face is recognized using encoding technique. In this technique the image is broken down into pixels of arrays.First the path 
directory of the image is given. Then the function looks through all the faces in the folder and returns the encoded faces. Only 
images stored in file format of .jpg and .png is acceptable. Thus the figure 4 shows the keys and the values of the encoded image. 


faces = get encoded faces() 
faces_encoded = list(faces.values()) 
known_face_names = list(faces.keys()) 


L.O333521935E-U2 ->.32L/00LIE-O2 -4./OOOL>20E-UL 5.4/0055/4e-02 
3.15563649e-61 -2.0786/7086e-61 2.31499985e-61 2.06786456e-01 
-4.37566265e-02 1.14323556e-01 7.05450028e-@2 1.383224/73e-01 
-1.88849958e-01 -5.01499099e-02 -1.90896854e-01 -1.27284288e-01 
-5.66866276e-@2 -1.04256186e-@1 -3.17125581e-64 1.63849143e-61] 

{‘bill gates’: array([-9.6699737@e-62, 7.6461777le-@2, 7.49358386e-02, 6.52936626e-@2, 
-1.30877569e-61, -1.64963630e-62, -4.42924462e-62, -1.38482258e-61, 
8.18997324e-02, -7.89919123e-02, 1.54310763e-61, -4.71304264e-03, 
-3.01/14987e-61, -4.53581586e-02, -2.309/31//e-02, 8.61688//8e-02, 
-1.34310298e-01, -3.61792808e-02, -1.34122640e-61, -1.39717687e-81, 
-7.92492852e-02, 2.711601786e-02, 38.87625143e-62, -5.51768355e-82, 
-1.08454362e-01, -2.28425652e-01, -1.18895646e-61, -1.33575395e-01, 
1.60208841e-61, -8.01836848e-62, 6.42742440e-62, 6.65364489e-02, 
~1.54251/54e-01, -7.3/0/6029e-02, -3.82892229e-02, 1.39695406e-01, 
-5./396210/e-02, -7.65046/766e-65, 2.11846560e-61, -5.9544568se-82, 
-1.53120235e-61, 2.86596082e-02, 3.69631797e-62, 2.69958539e-01, 

-45754197e-61, 1.92170348e-62, -2.80094258e-02, -5.61595996e-02, 

-s2o//981le-81, -2./65158/5e-01, /.1/3128098e-62, 1.2////8/4e-01, 

.37651026e-61, 6.52648956e-62, 1.06730439e-81, -1.582727238e-681, 

~51731193e-82, 1.79224446e-681, -1.38644412e-61, 1.660935316e-61, 

.26939352e-61, -1.48586253e-01, 3.77498940e-62, -1.93176232e-02, 

.23646341e-81, 3.80572714e-02, -4.09558378e-02, -6.45784438e-82, 

-26812169e-G1, -1.9824230/7e-601, 5.13286/795e-64, 1.269052138e-81, 

-21852982e-02, -1.33324265e-601, -3.56793600e-61, 2.23459247e-62, 

-99945259e-61, 1.84157744e-61, -9.02191848e-62, -4.47266586e-02, 

.70497790e-02, -4.97399867e-62, 1.29505256e-62, 3.46676633e-02, 

-33/1/95/e-02, -1.50001198e-601, -1.3s244/124e-01, 2.6508081/e-02, 

1.92666978e-61, -4.9912303de-602, 4.087868/770e-62, 2.18005313e-01, 


Figure 4: Encoded values 
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4.2. Face recognition using CNN: 
The output from the face recognized using CNN is shown in figure 5,6,7,8,9. 


4.2.1. Accuracy plot: 
On implementing this model we gota accuracy of 98.76% 
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Figure 5: Accuracy plot 
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4.2.2. Loss plot: 
On implementing this model we gota loss of 9.79% 
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Figure 6: Loss plot 


4.2.3. Model detection and prediction of a person: 
If the person is authorized then it will display the face of an authorized person under the green colour rectangle, else it will 
display unknown person in red rectangle. 





Figure 8 Model prediction of an unauthorized person 
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4.2.4. Database: 
Figure 9 shows how details are stored in the database. 
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